-
Notifications
You must be signed in to change notification settings - Fork 5.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rendering and recording mirrored video stream from camera. #1628
base: gh-pages
Are you sure you want to change the base?
Conversation
@stepancar I was wondering if you could add the work around to make MediaRecorder recorded video seekable part the of sample. Doing so would inform developers that implement recording of the issue and potentially serve as reference to push Chromium team reconsider the Won't Fix state of the bug (I can always dream) if all other non chromium browsers proves not having the issue. I would understand if you refuse, at least mentioning that recording file may not be seekable from this sample when using Chromium based browser can in my point of view go in the right direction and help the cause. Chrome related WontFix bug: Know working workaround: |
@hthetiot , I would be happy do that too, but when I get any comments from maintainers to make sure this sample makes sense |
@fippo , could you review it? Thank you |
Hello @fippo! May I ask you to review it/validate idea? |
let recorders = []; | ||
|
||
startButton.addEventListener('click', async () => { | ||
const stream = await navigator.mediaDevices.getUserMedia({ video: { width: 1280, height: 720 } }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hard coded camera size will fail on many device.
Edit: you do part bellow already:
Use video: true constraints and videoWidth/Height on video tag once metadataloaded to get size instead or VideoTrack width or height settings.
@hthetiot , do you know how to get a feedback from maintainer in this project? |
@stepancar apologies for the radio silence. The samples repository is effectively in maintenance mode and hasn't had a dedicated maintainer for a while. The use-case for the sample is interesting, one normally does not think much about this as most mirroring is done with CSS but that does not apply to MediaRecorder. But mixing mirroring with MediaRecorder and then providing two modes makes this way more complicated than it would need to be. It would probably fit better as a processor in the video-processing sample but that one is not using a worker and the code makes me cry. If you still see the slowdown filing a specific crbug for it would be better. |
@fippo , thank you for your answer! I know that usually people mirror video with css. This works well when we need to implement video calls app. This demo helps to better understand limitations of MediaRecorder API / canvas.captureStream and demonstrates how https://github.com/w3c/mediacapture-transform can help here. Yes, know about https://webrtc.github.io/samples/src/content/insertable-streams/video-processing/ I didn't get your point about crbug, could you elaborate on that? Media recorder behaviour follows specification. |
I get the use-case and it is great! But adding it to the simple recording demo is out of question because iirc Firefox does not support breakoutbox yet (and don't even try to get started with worker vs non-worker) and it brings in a ton of complexity. Now if you actually look at the problem that needs solving it would be way more efficient just to use libyuv's I420Mirror method in-place rather than doing multiple thread hops etc. Works nicely but requires a JS API change to add a flip option. Over to @guidou |
@stepancar Can you file a spec bug at https://github.com/w3c/mediacapture-main/issues/ or https://github.com/w3c/mediacapture-record/issues explaining the use case and, if you have a proposal, a sketch of it and its rationale? |
@fippo , I totally agree with you that it would be nice to have simple but the problem is that those api are not about transforming streams and introducing such option there is abstraction/responsibility leaking. Ideally, it should be another, non javascript api for making such transformations, without requiring to write js code and thinking about workers, etc. const mediaStream = new MediaStreamTransform(originalMediaStram, {
transform: 'scaleY(-1)', // https://developer.mozilla.org/en-US/docs/Web/CSS/transform
filter: 'blur(5px)' // https://developer.mozilla.org/en-US/docs/Web/CSS/filter
}) Talking about firefox is not supported. I'm not sure how this can affect this sample. this sample is for https://github.com/w3c/mediacapture-transform section. If it's not supported by vendor - it's expected that it would not work. |
Description
Added a demo which demonstrates rendering and recording mirrored video stream from camera.
Purpose
Mirroring video is a common case in video processing.
#1626
This demo shows advantage of processing video frame-by-frame in a worker
https://stepancar.github.io/webrtc-samples/src/content/insertable-streams/video-recording/